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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 
All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
of the Office or upon petition by the applicant. See 37 CFR 1.313 and MPEP 1308. 

1 . ^ This communication is responsive to the communication filed 22 October 2012 . 
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the restriction requirement and election have been incorporated into this action. 
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a) ^ All b) □ Some* c) □ None of the: 
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3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). 
* Certified copies not received: . 

Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 

5. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 
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EXAMINER'S AMENDMENT 

1 . An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1 .31 2. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with William Frommer on 26 October 2012. 

The application has been amended as follows: 

1 . (Currently Amended) An audience state estimation system comprising: 
imaging device for imaging an audience and generating a video signal 
relative to the audience thus imaged; 

movement amount detection device for detecting a movement amount of 
said audience based on said video signal, 

wherein the movement amount is determined from a magnitude of 
movement, periodicity of movement, a center of gravity of a power spectrum, a 
volume of sound, and periodicity of sound based on a contents provision state 
which indicates an environment condition of the audience that affects difficulty 
factors for obtaining movement of the audience and for obtaining sound of the 
audience, 

wherein the movement amount detection device discriminates and 
extracts a pixel range which is a flesh-color area identifying flesh color from said 
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video signal, divides the extracted flesh-color area into blocks, and calculates a 
movement vector for each of the divided blocks, 

wherein the blocks include a face block representing a face unit of the 
audience and a hand block representing a hand unit of the audience, and block 
matching of a current image and a next or previous frame image is performed for 
each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and each 
of the plurality of pixels identifies flesh color; and 

estimation device for estimating an audience state based on a comparison 
result of said movement amount and a predetermined reference level,, 

wherein said estimation device estimates said audience state to be in a 
state of beating time with the hands when said movement periodicity is larger 
than the predetermined reference level, and estimates said audience state to be 
in a state of clapping when said movement periodicity is not larger than said 
predetermined reference level . 



6. (Currently Amended) An audience state estimation system comprising: 
imaging device for imaging an audience and generating a video signal 
relative to the audience thus imaged; 
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movement periodicity detection device for detecting movement periodicity 
of said audience based on said video signal, 

wherein the movement periodicity is determined from a magnitude of 
movement, periodicity of movement, a center of gravity of a power spectrum, a 
volume of sound, and periodicity of sound based on a contents provision state 
which indicates an environment condition of the audience that affects difficulty 
factors for obtaining movement of the audience and for obtaining sound of the 
audience, 

wherein the movement periodicity detection device discriminates and 
extracts a pixel range which is a flesh-color area identifying flesh color from said 
video signal, divides the extracted flesh-color area into blocks, and calculates a 
movement vector for each of the divided blocks, 

wherein the blocks include a face block representing a face unit of the 
audience and a hand block representing a hand unit of the audience, and block 
matching of a current image and a next or previous frame image is performed for 
each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and each 
of the plurality of pixels identifies flesh color; and 
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estimation device for estimating an audience state based on a comparison 
result of the movement periodicity of said audience and a predetermined 
reference leveL 

wherein said estimation device estimates said audience state to be in a 
state of beating time with the hands when said movement periodicity is larger 
than the predetermined reference level, and estimates said audience state to be 
in a state of clapping when said movement periodicity is not larger than said 
predetermined reference level . 



11. (Cancelled). 



29. (Currently Amended) An audience state estimation system 
comprising: 

input device for inputting and generating at least one of video signal 
obtained by imaging an audience and audio signal obtained according to sound 
from said audience; 

characteristic amount detection device for detecting, based on said video 
signal, at least one of a movement amount and movement periodicity of said 
audience, and for detecting, based on said audio signal, a piece of information on 
at least one of a volume of sound from said audience, periodicity of said sound, 
and a frequency component of said sound, 
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wherein the at least one of a movement amount and movement periodicity 
is determined from a magnitude of movement, periodicity of movement, a center 
of gravity of a power spectrum, a volume of sound, and periodicity of sound 
based on a contents provision state which indicates an environment condition of 
the audience that affects difficulty factors for obtaining movement of the audience 
and for obtaining sound of the audience, 

wherein the characteristic amount detection device discriminates and 
extracts a pixel range which is a flesh-color area identifying flesh color from said 
video signal, divides the extracted flesh-color area into blocks, and calculates a 
movement vector for each of the divided blocks, 

wherein the blocks include a face block representing a face unit of the 
audience and a hand block representing a hand unit of the audience, and block 
matching of a current image and a next or previous frame image is performed for 
each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and each 
of the plurality of pixels identifies flesh color; and 

estimation device for estimating an audience state based on a comparison 
result of the detected result of said characteristic amount detection device and a 
predetermined reference leveL 



Application/Control Number: 10/602,779 Page 7 

Art Unit: 2427 

wherein said estimation device estimates said audience state to be in a 
state of beating time with the hands when said movement periodicity is larger 
than the predetermined reference level, and estimates said audience state to be 
in a state of clapping when said movement periodicity is not larger than said 
predetermined reference level . 

31 . (Currently Amended) An audience state estimation method 
comprising: 

imaging an audience and generating a video signal relative to the 
audience thus imaged; 

detecting a movement amount of said audience based on said video 

signal, 

wherein the movement amount is determined from a magnitude of 
movement, periodicity of movement, a center of gravity of a power spectrum, a 
volume of sound, and periodicity of sound based on a contents provision state 
which indicates an environment condition of the audience that affects difficulty 
factors for obtaining movement of the audience and for obtaining sound of the 
audience, 

discriminating and extracting a pixel range which is a flesh-color area 
identifying flesh color from said video signal; 

dividing the extracted flesh-color area into blocks; 
calculating a movement vector for each of the divided blocks, 
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wherein the blocks include a face block representing a face unit of the 
audience and a hand block representing a hand unit of the audience, and block 
matching of a current image and a next or previous frame image is performed for 
each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and each 
of the plurality of pixels identifies flesh color; and 

estimating an audience state based on a comparison result of said 
movement amount and a predetermined reference leveL 

wherein said audience state is estimated to be in a state of beating time 
with the hands when said movement periodicity is larger than the predetermined 
reference level, and said audience state is estimated to be in a state of clapping 
when said movement periodicity is not larger than said predetermined reference 
level . 

35. (Currently Amended) An audience state estimation method 
comprising: 

imaging an audience and generating a video signal relative to the 
audience thus imaged; 
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detecting movement periodicity of said audience based on said video 

signal, 

wherein the movement periodicity is determined from a magnitude of 
movement, periodicity of movement, a center of gravity of a power spectrum, a 
volume of sound, and periodicity of sound based on a contents provision state 
which indicates an environment condition of the audience that affects difficulty 
factors for obtaining movement of the audience and for obtaining sound of the 
audience, 

discriminating and extracting a pixel range which is a flesh-color area 
identifying flesh color from said video signal; 

dividing the extracted flesh-color area into blocks; 

calculating a movement vector for each of the divided blocks, 

wherein the blocks include a face block representing a face unit of the 
audience and a hand block representing a hand unit of the audience, and block 
matching of a current image and a next or previous frame image is performed for 
each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and each 
of the plurality of pixels identifies flesh color; and 
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estimating an audience state based on a comparison result of the 
movement periodicity of said audience and a predetermined reference leveL 

wherein said audience state is estimated to be in a state of beating time 
with the hands when said movement periodicity is larger than the predetermined 
reference level and said audience state is estimated to be in a state of clapping 
when said movement periodicity is not larger than said predetermined reference 
level . 

38. (Cancelled). 

55. (Currently Amended) An audience state estimation method 
comprising: 

generating any one of a video signal obtained by imaging an audience and 
an audio signal according to sound from said audience; 

detecting, based on said video signal, at least one of a movement amount 
and movement periodicity of said audience, 

wherein the at least one of a movement amount and movement periodicity 
is determined from a magnitude of movement, periodicity of movement, a center 
of gravity of a power spectrum, a volume of sound, and periodicity of sound 
based on a contents provision state which indicates an environment condition of 
the audience that affects difficulty factors for obtaining movement of the audience 
and for obtaining sound of the audience, 
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discriminating and extracting a pixel range which is a flesh-color area 
identifying flesh color from said video signal; 

dividing the extracted flesh-color area into blocks; 

calculating a movement vector for each of the divided blocks, 

wherein the blocks include a face block representing a face unit of the 
audience and a hand block representing a hand unit of the audience, and block 
matching of a current image and a next or previous frame image is performed for 
each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and each 
of the plurality of pixels identifies flesh color; 

detecting, based on said audio signal, a piece of information on at least 
one of a volume of sound from said audience, periodicity of said sound, and a 
frequency component of said sound; and 

estimating an audience state based on a comparison result of said 
detected result and a predetermined reference level., 

wherein said audience state is estimated to be in a state of beating time 
with the hands when said movement periodicity is larger than the predetermined 
reference level, and said audience state is estimated to be in a state of clapping 
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when said movement periodicity is not larger than said predetermined reference 
level . 

57. (Currently Amended) A non-transitory computer-readable medium 
storing an audience state estimation program, executed by a computer- 
processor, for estimating an audience state by processing information, said 
program comprising: 

a step of performing any one of detection, based on said video signal 
obtained by imaging the audience, for at least one of a movement amount and 
movement periodicity of said audience, and detection, based on said audio signal 
according to sound from said audience, for a piece of information on at least one 
of a volume of sound from said audience, periodicity of said sound, and a 
frequency component of said sound, 

wherein the at least one of a movement amount and movement 
periodicity is determined from a magnitude of movement, periodicity of 
movement, a center of gravity of a power spectrum, a volume of sound, and 
periodicity of sound based on a contents provision state which indicates an 
environment condition of the audience that affects difficulty factors for obtaining 
movement of the audience and for obtaining sound of the audience, 

wherein the step of performing detection discriminates and extracts 
a pixel range which is a flesh-color area identifying flesh color from said video 
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signal, divides the extracted flesh-color area into blocks, and calculates a 
movement vector for each of the divided blocks, 

wherein the blocks include a face block representing a face unit of 
the audience and a hand block representing a hand unit of the audience, and 
block matching of a current image and a next or previous frame image is 
performed for each of the blocks, 

wherein the movement vector is the movement direction and the 
movement amount when a result of the block matching indicating images of the 
blocks are matched, 

wherein each of the divided blocks includes a plurality of pixels, and 
each of the plurality of pixels identifies flesh color; and 

a step of estimating the audience state based on a comparison result of 
said detected result and a predetermined reference leveL 

wherein said audience state is estimated to be in a state of beating time 
with the hands when said movement periodicity is larger than the predetermined 
reference level, and said audience state is estimated to be in a state of clapping 
when said movement periodicity is not larger than said predetermined reference 
level . 



Allowable Subject Matter 

2. Claims 1 , 2, 4-10, 29-37, and 55-58 are allowed. 

The following is an examiner's statement of reasons for allowance: 
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The examiner concludes that the prior art does not teach alone or in combination 
all the limitations of claims 1 , 2, 4-10, 29-37, and 55-58. In particular, the prior art does 
not teach "wherein the movement amount is determined from a magnitude of 
movement, periodicity of movement, a center of gravity of a power spectrum, a volume 
of sound, and periodicity of sound based on a contents provision state" in combination 
with the limitation "wherein said estimation device estimates said audience state to be in 
a state of beating time with the hands when said movement periodicity is larger than the 
predetermined reference level, and estimates said audience state to be in a state of 
clapping when said movement periodicity is not larger than said predetermined 
reference level" and other limitations as stated in claim 1 and similarly in claims 6, 29, 
31 , 35, 55, and 58. Therefore, claims 1 , 2, 4-10, 29-37, and 55-58 are allowed. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JEREMY DUFFIELD whose telephone number is 
(571)270-1643. The examiner can normally be reached on Mon.-Fri. 8:00 A.M. -5:30 
P.M. EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Scott Beliveau can be reached on (571) 272-7343. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



26 October 2012 

/Jeremy Duffield/ 
Examiner, Art Unit 2427 

/Scott Beliveau/ 

Supervisory Patent Examiner, Art Unit 2427 



